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COMMUNITY COLLEGE INSTITUTIONAL RESEARCH: WHAT. WHY, AND KOW? 



In business, research and development (R&D) is usually the 
department last to be funded and first to go in times of budgetary 
crisis. The same is the case of community college institutional 
research (abbreviated IR) . Anthony Zeiss, President of Pueblo 
Community College, Colorado, notes, "Of the 1,221 public community 
colleges in this country, it is estimated that only 60 percent have 
designated full-time institutional research persons (AACJC, 1985) , 
yet as a relatively new system of higher education we have a 
tremendous need to understand what we are doing well, where we can 
improve, and what we should be anticipating in the future." (Zeiss, 
1986, p. 37) In a local study of the 9 colleges in the Los Angeles 
Community College District, Stuart Wilcox states that "The 
certified research staff totaled to the equivalent of 7.25 full- 
time personnel, averaging 0.81 researchers per college." (Wilcox, 
1982, p. 2) Obviously, IR is in general an area of few staff and 
monetary resources . 

Decision-makers, as well as the researchers themselves, are 
concerned with the type , validity, and usefulness of the data 
collected by the IR offices. In the survey mentioned above, Stuart 
Wilcox concludes that about half an IR office's time is spent on 
self studies, that is, the identification of trends within the 
student population (demographics, program enrollment changes , grade 
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distribution, etc.), follow-up studies of graduates, and evaluation 
of programs and procedures. He also notes that, in the opinion of 
the researchers, more time and funds should be spent on just these 
types of activities. External reporting, such as filling out 
questionnaires, actually takes up only 9% of the IR office's time. 
(Wilcox, 1982) 

One question then becomes: what can and should an IR office 
be doing? Clearly, in order to fulfill all the desired functions 
of such an office, more resources need to be allocated to this type 
of endeavor. Moreover, what is regarded as useful research changes 
with the preoccupations of different time periods. According to 
Edith Carter, the 1960s were characterized by an emphasis on the 
reporting of numbers of students enrolled, and growth in quality 
was seen as an end in itself. The 1970s saw a fine-tuning of the 
data-collection methods due to reporting requirements by state and 
federal funding and regulatory agencies and a concern with the 
impact of the emerging technologies. With stabilizing and 
occasionally declining enrollments of the early 1980s came a focus 
on impact studies, marketing, budgets, and fundraising, with a 
special emphasis on accountability. (Carter, 1986, pp. 85-86) In 
terms of what institutional research should be doing. Carter also 
dares to make the following controversial statement , "There is a 
growing concern that society can no longer afford education for the 
masses. The theme of the open door is being replaced with those of 
limited access, quality, and excellence. The concept of 
quantitative growth has been replaced with that of qualitative 



growth." (Carter, 1986, p. 86) Some of these qualitative studies 
are beginning to appear in the literature, but they are more 
written about than carried out, for reasons shown below The types 
of research currently being done may be divided into the following 
four main areas according to the purpose of the studies: external 
reporting, self-improvement (formative research) , program decision- 
making (essentially cause/effect) , and public relations and 
marketing. This paper will attempt to describe the components of 
these four areas, draw conclusions concerning appropriate standards 
for the different types of research, and conclude with a short 
analysis of six research reports from formative and marketing 
studies . 

I . Interconnecting areas of research 

Since institutional research is carried out at an individual 
college campus, it has frequently been termed "self study." (See 
Blai, 1982) Although certain theoreticians would thereby 
automatically cast doubt on the findings of such studies, 
suspecting that they were tainted by investigator bias (See e.g. 
Scriven, 1983), there is little av^ubt that in the day-to-day 
management of a college as well as in long-range planning, self 
study is necessary for such a multi-million dollar business to 
thrive in today's environment of tightening purse-strings and 
public requirements of accountability. It must be understood that 



in segmenting out areas of research, I am merely presenting one set 
of eye-glasses though which to look at institutional researcl^r 
there are many others. In a certain way, all research data can be 
utilized fcr formative purposes, that is, to compare how what one 
has measures up to one's standard of a desired state, be that 100% 
transfer of students to full universities, 100% student 
satisfaction with the educational experiences they are being 
offered, or some other criterion. One may also say that all data 
may be used for decision makirg: by the college administration 
locally, or by state and federal agencies. The data collected can 
be used for a variety of purposes, and for these diverging purposes 
different criteria of acceptability may be employed. Good research 
depends as much on what question is asked as on how the data are 
gathered and reported. 



II- Areas of Institutional Research 

A. External reporting and policy influencing purposes 

"For many years, our colleges have suffered because policy 
makers were making intuitive judgments about our programs and 
students. Their conclusions were, at times, incorrect but 
convenient for a particular philosophy or preconceived notion. 
Their actions were usually unsupported by any real evaluation 
data." (George R. Boggs in the preface to the California 
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Association of Comnmnity Colleges [CACC] Commission on Research 
Report, 1988) In response to a decade of intense public scrutiny 
of the community colleges througli questions relating to their 
accountability as publicly funded learning institutions , 
institutional research is now turning to not only supplying the 
question-askers with data, but also seeking to shape the policies 
of those political decision making bodies . Community college IR 
offices at the behest of their college presidents, CEOs, and boards 
are taking a proactive stance in addition to the necessary reactive 
mode of operation. Also according to the CACC report, 
••Institutional research provides external benefits by demonstrating 
to the general public, the legislature, and to others at the state 
level, that community college programs are effective, that students 
and society alike benefit from them, and that they are a wise 
expenditure of public funds." (CACC, 1988, p. 3) One may wish to 
be somewhat wary of such a statement, however, since it may lead 
to the invalid research strategy of coming up with conclusions in 
advance and then selecting data to fit these. Researchers as well 
as decision makers must be cautious of the tendency to see only 
what they want to see. 

With the individual state providing more and more of the 
financial support of community colleges and making a significant 
number of policy decisions affecting these, there is now an 
increasing awareness of the importance of a good liaison with 
appropriate state agencies. According to Ivan Lach, "There has been 
a dramatic growth in the demand for information about cominuni ty 



CO] lege students , ins true tional programs , facilities, and personnel 
by the various state agencies that have a grfraf impact on policy 
formulation for community colleges." (Lach, 1981, p. 7) The state's 
concern is to develop policy regarding all the coipmunity colleges 
within its borders, and for this reason community college IR often 
falls short of providing the kind of information needed - 
Cooperative and joint research efforts among all the s tate ' s 
community colleges is necessary to influence policy makers 
appropriately; however, the fact of the great diversity among 
individual community colleges makes this type of research extremely 
difficult. We know intuitively, and to a certain degree from 
research by outside critics, that urban colleges vary considerably 
from rural ones in the demographics of their student body, their 
objectives, their organization, and their history. 

Influencing policy is also a concern regarding the community 
college's relationship with the federal government, and the same, 
if larger, caveats apply as with the state government. In the words 
of Cheryl Opacinch, "Federal postsecondary policy encompasses three 
major types of programs: (1) financial assistance programs in the 
form of grants and loans for graduate and undergraduate students 
that are designed to increase access to higher education, (2) 
institutional building programs that assist colleges in facilities 
building, and (3) categorical programs that provide assistance to 
improve the quality of instruction in general, or for a specialized 
clientele." (Opacinch, 1981, p. 18) Obtaining federal grant money 
is big business for all ins:t itute*^ of higher t^ducatiop, but since 



community colleges have focused on teaching rather than theoretical 
research, they have not gone after the federal dollars with the 
same zeal that universities do. Although the availability of 
federally guaranteed student loans nay be of importance in 
attracting students, state policies impinge on tlie situation of the 
community college far more and so has been focused on to the near 
exclusion of the relationship with tlie federal government. 

One aspect of data-gathering frequently emphasized by outside 
researchers studying community colleges has been the function of 
transfer to four-year institutions. It seems, however, that this 
type of study is fraught with a certain def ensiveness by the 
community colleges' institutional researchers, and very few reports 
have been published that try to get to grips with the problem. In 
the author's opinion, it would behoove the community college 
leadership to take the bull by the horns and engage in public 
debate of a definition of the concept of transfer, its priority in 
the set of college goals, and serious discussion of the realism 
behind an expectation that any institute of learning can take an 
underprepared student and, within the current fiscal constraints, 
turn her/him into a transferable college student. Richard Rinehart , 
in his article "Assessing successful articulation of transfer 
students," (Rinehart, 1977) has paid some heed to the transfer 
miasma, and the problem deserves more recognition by institutional 
research partly because it is used as the heaviest argument in 
giving community colleges failing grades as learning centers by 
outside critics. Two studies dealinn Kith transfer will be 
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discussed in the last section of this paper. 

In conclusion, the relationship between the outsidb- po'^itical 
world and the community college is a tvjo-way street: it influence? 
the college's programs and policies and the college may atteirpt to 
influence it. According to Warren Groff, "The external environment 
includes demographic trends, social indicators, economic trends, 
political change, technological advances, changes in the workplace, 
information technologies, value shifts, and numerous other 
variables." (Groff, 1986, p. 62) Institutional research must 
attempt to analyze current trends in all these areas and predict 
future populations if it is to be of value relative to external 
funding sources and critics. 

B - Institutional research for self -improvement 

The bulk of published studies of commuaity college 
institutional research concerns the importance of formative 
research to the individual college. There is considerable 
duplication of effort in this area, not sui. isingly, perhaps, 
since each community college tends to see itself as unique and that 
studies conducted at other colleges can only with difficulty be 
generalized to be relevant to it. 

Of the many sub-purposes of formative research, I would here 
like to consider the following: efficiency studies, effectiveness 
studies, needs assessnents, and faculty/staff assesr.mri^nf s . I wonld 
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also like to separate out research geared toward docision making 

into a category by itself because I feti] that this is at a 

level 

different conceptual^ than the above: decision leaking is tlie 
application of research rather than a primary purpose for it. 

1 ) Efficiency studies 

Efficiency studies concern the cost of a teacher-student 
ratio, the employment of full-time vs. part-time faculty, the use 
of facilities, energy usage, class offerings and cancellations, 
etc. To determine a level of efficiency, one must compare the 
factual findings to a stated criterion, e.g. graduation or transfer 
rates. William Kennedy of Cuyahoga Community College puts it in 
these terms, "Under the pressure of the budget crunch that many 
two-year colleges are now experiencing (and will undoubtedly 
experience to a greater degree in the 1980s) , many decision makers 
are looking to make hard decisions that will impact their schools 
for several years by utilizing a cost-effectiveness model. This 
type of evaluation document will frequently contain such breakouts 
as input variables, process variables, output variables, and 
outcome variables, content variables, contextual variables, etc." 
(Kennedy, 1981, p- 2) The focus of these studies are the questions: 
are we using our resources to their fullest potential? Compared to 
our goals and our mission, can we use them more effectively? Where 
should we put our major effort in requesting funding: in buildings, 
equipment, teachers, staff, materials, new proarans . etc.? 



Financial planning has a large role in the area of efficiency in 
time? of budget cuts and declining enrollment . Stevenson and 
Walleri note, ''Financial decisiv'^n malting includef? rosourc^i 
allocation, utilization^ and accountability. This alsc includes the 
processes of planning, budgeting, spending, and accounting or 
evaluation. (Stevensoii and Walleri, 1981, p. 83) Since answers to 
questions of resource allocations are mostly relevant to individual 
colleges, institutional research 5s generally commissioned with the 
gathering of appropriate data for such purposes. 

2 ) Effectiveness /outcome studies 

In addition to using its resources efficiently, community 
colleges are increasingly being held accountable for producing a 
measurable effect of learning in its students. Institutional 
research is needed to determine what these effects are and whether 
and to which degree they meet expectations . A variety of methods 
are used, rhe most frequent being standardized tests of several 
domains of learning, observations, longitudinal follow-up studies 
of students' performance in other educational institutions or in 
the job marketplace, and teacher rating of outromes. According to 
Kay McCullough Moore of Alamo Community College District in San 
Antonio, Texas, "The major challenge now is to discover meaningful 
and useful v^ays of distinguishing between effective and ineffective 
institutions, rrograjns, processes, and even people." (McCullough 
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Moore, 1986, p. 51) A crucial question becomes what constitutes 
reasonable measures of of f ect i vent^ss considt^rina com^^ainity 
colleges' unique ''open door" access pclicier. Community colleger 
(unlike four-year institutions and uiiiv* rsities) cannot T7a^'^ away 
the problem by selecting only the top students who are sur? to 
succeed nc matter what they are given by way of cciu'^ation. Fcr this 
reason, a "value added" approach to measuring learning has been 
adopted by many schools. In simple terms, the "value added" concept 
consists of pre-testing, post-testing, and determining the amount 
of "added value" that the school has been responsible for creating 
in the students. As Oscar Lenning states, quoting Hart (1971), " 
Institutions should be judged not by their outputs alone, but by 
their outputs relative to their inputs." (Lenning, 1977, p. 15) A 
student who comes to the college reading at the seventh grade level 
may exit a course or a year reading at a 12th grade level, while 
another who comes in reading at a 10th grade level may only advance 
to 13th. One would then conclude that the former student had had 
more value added than the latter even though his/her score was 
actually lower. Naturally, there are many pitfalls in such 
research, the primary being the attribution of cause and effect. 
Researchers are aware that there may be a host of external 
circumstancer. influencing test achievements , and perhaps only a 
small part of the improvement can be attributed to the influence 
of the school. However, careful planning and in-depth knowledge of 
research and evaluation issues can at least help control for such 
confoundina far^ ors. 

ERIC 



standardized achievement testing is not, of course, the only 
method used in higher education to assess learning. Criterion- 
referenced tests have become increasingly popular through the 1970- 
and 1980s. (See Lenning 1977, p. 12) In addition, observation of 
simulations is another way to assess learning, a very promising 
one, but one which requires a high level of observer coTT^pe t ence , 
problems of inter-rater reliability, etc. that may be beyond the 
scope of semester-end community college assessments. 

Most community colleges offer a variety of different prograins: 
vocational (unfortunately called "terminal") programs at different 
levels of complexity, remedial programs, traditional liberal arts 
programs, and a world of different not-f or-credit courses. All of 
these expect learning to take place, but they need to be assessed 
in different ways depending on the type of learning emphasized. 
Valid evaluation for these must take into account both w)iat the 
students can and cannot do upon entry and in what domain the 
learning is expected to occur. Is it purely academic? A 
standardized test might do. It is skills-oriented? Then a 
demonstration of those skills would have to be a requirement for 
course exit. Is social awareness and attitude (perseverance, 
values, etc) included in the program objectives? Then these too 
would have to be assessed. No one instrument or approach can be 
said to serve for all the purposes of all the programs offered at 
a community college. As McCullough Moore states, " Because 
effectiveness is mult idiniensional and educational outcomes arc- 
multiple and diverse, it must b^^ obvious that th^a^c ran be n^^ 



single criterion for institutional effectiveness.** (McCullough 
Moore, 1986, p. 54) Institutional research of outcori»es miis^ 
take into consideration the philosophical question of th^? extend 
to which any educational program can contribute to the social 
mobility of its customers . Brent Shea adds the admonition , **The 
exclusive concern with academic achievement has resulted in the 
reported failure of previous innovative programs, owing to their 
lack impact on of cognitive development." (Shea, 1977, p. 82) Quite 
simply, a program^ total objectives in all domains must be made 
clear before any evaluation of its impact can validly be made. 
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3 . Needs assessment studies 

Essential to the formative evaluation of community college 
programs is the question: who are our present and potential 
students? Studies in this area concern population demographics, the 
stated purposes of attending by students, their entering skill 
levels , etc . The formative purpose is : how can we meet our 
population's needs better? Taking one step backwards in determining 
needs is the acknowledgment that any declaration of a need is based 
on a set of values. To put it somewhat facetiously: we find that 
the majority of our constituents cannot read Ulysses with a high 
degree of comprehension, nor can they carry out CPR. Does that 
automatically constitute a need? Any need must be seen relative to 
a goal and the value of that goal . 
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So-called needs assessments are carried out by at least three 
different groups for divergent purposes. The first is the one we 
are concerned with in this article: higher education institutional 
research offices. They generally want to discover how many people 
in their locality want what they have to offer and what this 
interest-group is like. This may be seen as partly marketing and 
partly prograr planning. Secondly, state and federal planning 
offices conduct needs assessments to ensure more or less equal 
access to education within their jurisdiction. Thirdly, external 
researchers at colleges and universities conduct demographic and 
interest surveys for their own research needs. One may safely say 
that there is no dearth of demographic data available; the problem 

a 

may lie more in the distinctions maJfe in the categories of 
respondents and the interpretations of the results. The national 
census produces volumes of demographic data, the National Center 
for Education Statistics is a reliable source, ancl, in addition, 
state planning offices supplement the colleges* own local surveys 
in describing the community characteristics. For the interest of 
the individual community college, however, the most important step 
in considering another needs assessment is to clearly define its 
need for information. Patricia Cross, in a highly informative 
article, states that the needs assessor could be more successful 
by obtaining information that "helped determine how a course should 
be structured, how and where it should be advertised to reach the 
target population, when and where it should be offered to provide 
convenience to the largest number of potential students, what 
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auxiliary services would be required by students, and what other 
courses they might take concurrently with (the course ir 
question)." Cross, 1979, pp. 197-198} The best r^oiriruni 1 y coll^ap 
needs assessjnent s do just this. Cross also alerts us to the 
importance of attempting to plan for future needs . The CACC report 
on institutional research asks these questions: "What are the 
important demographic, economic, and social trends in the state and 
in the communities that will be affecting community colleges? What 
will be the nature of the colleges' communities in the future? Who 
will be the potential students?"CCACC , 1988, p. 3) Trendfs analyses, 
understanding of the marketplace, and a good dollop of visionary 
enthusiasm are needed to try to plan for the future and not only 
fight the fires of today, 

4 . Faculty and staff development 

Due to the forces of the employment marketplace, this area of 

research does not seem to produce a large share of the articles 
r 

written about of by community college institutional research. There 
are many more qualified teachers than the colleges can employ, and 
with current systems of collective bargaining and tenure, there is 
little incentive for community college faculty to take a great 
interest in self -improvement . Secondly, for financial reasons, many 
community colleges utilize a heavily part-time faculty and spending 
money on their improvement seems to bn^ e^conomically self-defeating. 
However, the passing of Assembl\ Pil] 27^'" ir C^]:f'^rnia 



delineating a future, phased-in requirement of a full-tiro to part- 
time faculty ratio of 75% to 25%, the author hold eves that th^r-^ nrea 
of staff development as much as sraff assessir.ent vill pl?y a larger 
and larger part in institutional research. Volume 41 of the r^eries 
New Directions for Community Colleges is devoted to the evaluation 
and development of staff and faculty and contains several pertinent 
articles. In one article, the volume editor Al Smith discusses 
seven distinct programs for faculty evaluation devised by different 
community colleges, and he concludes, "The shift to staff 
evaluation should enhance employee morale, job satisfaction, and 
staff productivity. Evaluation should enable employees to see ways 
to grow without leaving their current jobs. This should be a 
healthy development in a profession where there is currently little 
opportunity for job mobility either externally or internally." 
Smith, 1983, p. 8) 

William Cashin is concerned with the use of student ratings 
as measures of teacher quality. He states, "Many two-year 
institutions do not have merit pay systems, academic rank, or even 
tenure. At these institutions, retention or termination is the only 
real personnel decision." (Cashin, 1983, p. 59) He discusses the 
importance of the readability level of the rating forms as possible 
causes of mis-rating by students, but he also adds, "If one is 
willing to accept ratings by colleagues or administrators as a way 
of validating student ratings , studies have found cor r el at ions 
between student ratings and colleague or administrator ratings to 
range from .43 to .69." (Cashin, 198?, p. Gl) Thc^ .^r^a rf th.. ^'yvl 
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of evaluation instruments (r-^ploy and thr- uf^e':: tr vhlcli they 
should be put ir largely uncxardned and should be fruitful f^r 
insti tutional res- arch f":"^riC-_»r^.<^ . 



C- Decision making for prcgrar^ developinent and change 

It is with difficulty that I create a separate category of 
"research for decision making." As stated in the introduction, 
decision making comes as a result of all the above types and 
purposes of research. According to William Ramsey, "Research can 
be defined as a process to help institutions define their goals and 
purposes, identify their programs and policies that meet these 
goals, evaluate programs and find out whether they are useful to 
the institution, and chart the flow of resources to determine the 
cost effectiveness of the programs. "(Ramsey, 1981, 9. 26) The 
major divisional areas are in the distinction of short-term studies 
for immediate decision making versus long-term studies for future 
planning . 

Mantha Mehallis classifies the areas of institutional research 
relevant for college administration decision making into the 
following categories: 

Professional development (See faculty /staff development above) 
Adminis trativtr appli cat ion? (See efficiency/ effectiveness 
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studies above) 
Management (See efficiency studies above) 

Planning (See effectiveness and needs assesi-rren! studier; 

above ) 

Evaluation (See faculty/staff development and effectiveness 

studies above) 

Values clarification (Not mentioned in my categories - I 
consider this to be an aspect cf criteria used 
rather than an area of research) 



(Mehallis, 19S1, pp. 95-104) 



Our times are characterized by public demands for 
accountability in education and the coupling of funds tied to 
demonstrated program effectiveness. This is not to be rued, but to 
be welcomed as an opportunity to improve ourselves in delivering 
the product we say we can deliver. Warren Groff puts the situation 
in these words, "American managers in business and education are 
trained primarily in directing dynamic growth, measured in 
quantifiable variables - full-time equivalent (FTE) students, head 
counts, size of mainframe, number of terminals per FTE, and so on. 
Many schools and colleges have been trimming the fat, and scjne have 
gone into bankruptcy. Managing the downside is a much tougher task 
than directing dynamic growth. In the long run, this furm of 
natural selection can produce a stronger American education and 
training industry.'* (Groff, 1986, pp. 71-72) 
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A considerable number of studies are coming out in the area 
of integrating IR into the decision iraking process. Miami-Dade 
Community College appears to be a center for such activity. Tt/^c cf 
their papers to be recommended are found on the ERIC database (the 
Junior College Clearinghouse) : "Integrating Research into Decision 
Making" (ED 239 674) written by John Losak and Cathy Morris. 
Another is authored by Ted Wright and entitled, "Who Uses 
Institutional Research and Why?" (ED 267 879) Both studies (and the 
majority of other studies) found that perseverance rates (retention 
and attrition) are the areas of study most requested by college 
administrators. Whether this is truly the area that the 
administrators should focus on is open to debate, and some 
investigators have made recommendations that the institutional 
researchers ought to report directly to the college presidents and 
be included in shaping policy. This may or may not interfere with 
the concept of the impartiality and objectivity of research; some 
theorists would say it does (e.g Michael Scriven) , other would say 
that impartiality is less important that gathering information that 
is likely to actually be put to use by decision makers (e.g. 
Cronbach, Stufflebeam, Alkin, Cooley, Stake, and others) . 

D. Public relations and marketing 

The major aspect of marketing research carried out by IR 
offices consists of market surveys and are frequently 



indistinguishable from the needs assessirents noted above. Phone 
surveys and questionnaires are employed, often rather hurriedly put 
together and with a definitely informal tone. Community cclleges" 
utilize all channels of coir.nunication with the public in attempting 
to create a higher awareness of their offerings: radio, television, 
the print media , and f liei s and brochures in mass mai ling3 or 
targeted to special interest groups. Some of the so-called surveys 
are little more than "sunshine reports" intended to furnish only 
happy numbers of community ax^areness of the college for release in 
the local press. However, with the cross-fire tha^ community 
colleges have been subi^ctcd to in the last decade, it is hardly 
surprising that some occasionally feel the need to fire back. 

Public relations writing is often couched in battle terms: 
first line of offense, targeting the group, strategic marketing, 
etc, William Harper proposes three areas of public relations to be 
of interest in an effective PR program: internal communications, 
external communications, and development, (Harper, 1977, p. 3) 
Slocum and Johnson state very clearly, "The marketing of a college 
is very much like the marketing of any other product - it includes 
encouraging consumers to choose that product rather than doing 
without it or selecting a competitor." (Slocum and Johnson, 1977, 
p. 73) This may be disturbing reading to some: should one college 
compete with another when both of them are receiving state funds 
for their operations? Should our tax money be spent advertising 
education? On the other hand, if it is not advertised, would only 
those with high resources and access to information networks be the 



ones to take advantage of the college's offerings? 

The concept of "truth in advertising" is ctirrently being 
argued in all sectors of American business. In order to ensure that 
con^munity college marketers better understand the college's 
strength and functions. Kelly and Otto recommend the close 
relationship and constant communication between college 
administration, institutional research personnel, and the public 
relations office. They declare, "The health and vitality of an 
institution is to a large extent directly proportional to the 
integration of the comfort and confidence levels of those selling 
the carpus, those conducting research about the campus, and those 
leading the campus. - (Kelly and Otto, 1986, p. 76) The thrust of 
this statement is corroborated in most of the literature regarding 
the employment of institutional research into public relations 
efforts • 



III- Standards: What Constitutes Good Institutional Research? 

The standards to be noted here arise directly from the 
descriptions of the research conducted as set out above and from 
accepted research guidelines. First of all, all studies must follow 
a :ceptable sampling, statistical, definitional, and evr. .ve 
procedures. As an example, we cannot talk about transfer without 
defining what transfer means and making sure that the data we use 



to support our statements have been coDlected cn the basis of tht. 
saipe definition. Sirnilarly, if a telephone survei' is to have any 
validity, the non-response- rate must be p.ade explicit, zind . for 
even better use of the results, some investigation into the causes 
of non-response must be conducted. 

In addition to these general standards applying to all 
research (those of validity, reliability, and objectivity) , the 
concepts of practicality and timeliness are especially important 
in institutional research. The research findings must be presented 
to the decision makers in useful formats {;fed Wright, 1985, on the 
formats preferred by administrators) , and they must be available 
when they are needed, not the semester after. This is good 
research . 

If we break down the IR by purpose as I have done above, we 
may see some refocusing on certain elements of the criterion list 
depending on the types of research. For the sake of collecting 
enrollment and other facts for reporting purposes, the data must 
be complete, appropriately categorized, and any discrepancies 
accounted for. The source of the data must be examined: is the 
enrollment form or questionnaire readable at the student's level? 
Does it ask the right questions? Are any important questions left 
unasked? Efficiency data must be compared to a standard of 
effectiveness to have any meaning, i.e. not just the cost of the 
product, but the cost in relation to the goal and the quality of 
educational attainment. The interpretation of effectiveness studies 
is- essentially similar: in order tc drav7 conclusions concerning 



educational outcomes, one must clearly specify the cr iLoria used 
and specify arguments in favor of those criteria. Needs assessments 
and other present/potential student body de5:r:ript ic^ris^ rrust rtc>te 
sampling systeT., number of respondents, non-response r^te,'^, actuel 
questions, rationale for multiple-choice forrrats if ther^e ?.re Uf^ed, 
and the statistical procedures employed and frc::^ which one c'rav*^ 
conclusions. It is most certainly the business of the IP office to 
guarantee that appropriate research methods are employed in all its 
work. 
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IV- Samples of Institutional Research Studies 

A- Coast Community Colleges Community Telephone Survey 1980 
This survey was undertaken in order to assess: 

* the effectiveness of the District's marketing efforts 

* the community's perceptions of the relative importance of 

community college functions and services 

* the community's propensity to take part in District programs 
and services 

and 

* the educational interests and needs of the community. 

(Butler^ 1?S<^ intrrdiicticnj 
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As we can see, the purposes are those of marketing to present 
and future constituencies and an inforral self -asses^rerit of 
educational needs by respondents. Regarding sampling, the report 
states that "Within a 4% error margin, we are 98% confident that 
the 546 interviews from the survey accurately represent the 337,000 
adults (age 18 and over) that reside within the Coast Community 
College District." The District Research office has published eight 
reports based on the survey, all of which are available from the 
District office. (I have included numbers 1 and 8 in the list of 
references . ) 

According to the standards set out above, how does this study 
measure up? On the positive side, the actual questions spoken by 
the interviewers are given in Appendix A of report # 8. This form 
also shows clearly how an interviewer should skip questions if not 
relevant, how to record non-compliance or non-response, etc. The 
interviewer guidelines are excellent. Most of the questions are 
fairly straight-forward and the coding of the responses should not 
pose any major problem. However, there are two areas of concern 
with the survey: that the non-response rate is not given nor 
discussed (perhaps because the person picking up the phone did not 
speak English sufficiently to answer?) , and that no mention is made 
of the uses to which the survey answers were put. One certainly 
cannot require this in a research report, but it rrr1-n" nir would 
be enlightening to see. Other than this, it is a marketing survey, 
and the college adm.inistration will have to decide if it provider? 
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then with the information they need to make program decisions. 



B . Coirraunitv Needs Assessment 1987 

(Cerritcs Coririunity College District) 

This study was carried out to "assess the needs of che 
community with regard to educational programs and services provided 
by Cerritos College." The actual questions are included in the 
report, which is commendable • The individual researcher can then 
draw conclusions regarding the validity of each item, some of wh*' ch 
I will examine below. This study used randomly selected telephone 
numbers (as did the Coast study) , and noted that "Over seven 
thousand households were contacted resulting in 1,050' personal 
interviews." This is a much larger data base than that of the Coast 
study, but it is in fact only a response rate of about 14%. As in 
the Coast study, this needs assessment attempts to describe the 
characteristics of ♦'he respondents in the categories gender, age, 
ethnicity, household income, and 'Education level. The Cerritos 
study adds sections on a respondent's previous experience with the 
college . 

Other than wonder ':ig about causes ^f non-response, I'm 
concerned about the wording and interpretation of certain questions 
and answers . The first question asks whether the respondent has 
heard of Cerritos. Obviously, since the name has already been 
mentioned, and it is an institution of higher learning to brct:, it 
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is highly likely that rtost people would say yes. In fact, 95% of 
the resrondents did. Ho^'ever, they don't know much about the 
college: the admission requirements, when the fall semester starts, 
how much it costs, etc. Several questions ask the respondents to 
rate on a ten-point scale the importance of various college 
services and purposes. Naturally, the results show that everything 
is rated highly, and to distinguish between them is dubious use of 
statistics. Similar crit' que could be given to most of the survey 
items. The conclusion is clear: the wording of questions and 
responses solicited are crucial in creating a study whose findings 
are of practical value. 

C . Institutional Student Retention Progress Report, 
Gainesville College, 1987-88 

One sLudy that is not a "sunshine report" comes from 
Gainesville, Georgia, and is a study of the retention, attrition, 
and graduation rates of its student body. It is an open access 
school admitting, as they say "students with a wice range of 
academic experiences, abilities, achievement levels, interests, and 
goals." (p. 3) Demographic studies show that "according to the 1980 
census, the area which Gainesville College serves is considerably 
below the state of Georgia average in educational attainment." 
(p. 3) The study is massive and detailed. It concludes that its 
transfer rate is dismal r but adds that "retention should and will 
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be a byproduct of a quality educational program designed tc provide 
appropriate experiences tc facilitate the academic and porson:^! 
development of its students." {p. 25) Tlie report r?J:es a series of 
recommendations concerning the improvement of retention and 
achieving a racial balance similar to its service area, but it also 
notes that it cannot do magic. Providing the types of services that 
its clientele is willing to undergo is its main concern. 

This study is quite clear ar-d well founded although some of 
the data are rather small upon which to draw any conclusions. For 
example, on page 19 the study notes that two black students were 
admitted as full time first time freshmen in 1984 and that one had 
graduated at Gainesville by 1987, giving a graduation rate of 50%. 
If one reported only the 50%, it would indeed be misuse of 
statistics, but the report does include the actual number. My 
conclusion is that in spite of its small number base, this is a 
thorough report. 

D. Los Angeles Community College District Action Report 1987 

This IR report concerns formative issues of curriculum 
improvement at the L.A. Community College District campuses. The 
group of investigators developed a series of recommendations for 
action in 22 areas directly and marginally related to curriculum 
concerns, ranging from writing-across-the-curriculum to awareness 
of health care. The method consisted of e::pert opiiiion gathered 



from involved persons. All the persons comprising the group would 
be expected to have direct awareness of the current material , 
personnel, and level of instruction in the 22 arr^ar? and onv ir^y 
therefore consider their opinions to carry weight. The result of 
the study is an action-oriented package , with clearly stated 
divisions, goals, background information, problem statements, and 
specific recommendations. It ought to be a used and useful 
document, while not research in its traditional sense. 

E. The Los Rios Transfer Centers: A First Year Evaluation 
(August 1987) 

Another type of self -evaluation was conducted by the Los Rios 
Community College District (California) in 1987 of their transfer 
center program. This report evidences the necessity of the same 
caveats as other such studies intended for public dissemination: 
the fact that it is unlikely that the college would publish any 
findings that state dollars were unwisely spent. While not wanting 
to fault this particular study, the author finds that a dose of 
skepticism concerning the universally positive conclusions of this 
type of study is warranted. 

However, this study does what it intended to do. It polled 
1,400 students who had used the Transfer Centers during their first 
year of operation and got a response rate of 48.5%, quite 
respectable in questionnaire research . The que s t ions are those of 
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satisfaction (very good, good, average, etc.) and specific 
questions regarding an individual's transfer. Once again, one may 
quibble about the wordina of individual items, but this is 
generally so of all survey instruments. For example, question 8 
(What is your current educational status?) has the following answer 
choice available: "Transferred, but not attending now." I wonder 
what that option means to the person completing the survey . The 
statistical methods used are described and reasoned for, and the 
reasons for an adjusted response rate clearly given. It is a clear 
document and well presented. 



F. S.I.O. Student Information Questionnaire, Fall 19S4 

The San Francisco Community College District has conducted a 
questionnaire of its student body every other year since 1972. 
(Moss, 1986, p. 14) It is a decision-oriented research project 
intended to describe pertinent aspects of the students* daily lives 
in connection with their study at the District's Community College 
Centers (open access centers with no formal registration process) 
and at City College, a traditional community college. The S.I.Q. 
consists of a one page multiple-choice format questionnaire which 
can easily be adjusted to changes in the student population. 
However, as Judith Moss points out, one must be extremely careful 
in making changes as that would invalidate longitudinal studies. 
She also states regarding the value of the S.I.Q. , "The uses range 
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from serving as a reference to answering simple requests to 
influencing policy." (Moss. 19?6, p, 17) Some of these uses are: 
parking and transportat ic:: argunents and decisions, location of 
facilities, student servicer, and prog:-am planning and evaluation. 
(Moss, 1986, pp. 17-22) 

In the actual report of the S.I.Q. from 1984, Moss notes one 
important negative aspect of the questionnaire, namely that it is 
anonymous. She says, "A significant drawback in anonymous 
questionnaires is not having the ability to do any student follow- 
up, for example how a particular student's educational objectives 
have changed or how they have been met." (Moss, 1984, p. 1.1) This 
is indeed a concern, but asking students to write in their names 
or student numbers is thought to generate less valid and reliable 
data, and so one aspect of research is sacrificed due to the 
relevance of another. 



V. Conclusions 

There can be little doubt that IR fulfills an essential 
function at community colleges. Whether in times of plenty or times 
of scarcity, no rational management can exist without a fact-based 
in-depth knowledge of the past and present, and no reasonable 
planning for the future can occur without a factual basis on which 
to make projections. However, most researchers conclude that as it 
standr, there is too little emphasis on the IR activity. James 
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Wattenbdorger states, ''Studies founded upon institutional research 
have been almost universal that most colleges provide inadequate 
support and attention to the activity.** (Wat tenborger , 1983, p. 58) 
Richard Richardson adds his voice to the many desiring to set 
research agendas for community college IR, but he ends on a 
positive note: "The most promising sign currently on the horizon 
involves the growing awareness on the part of community college 
leaders of the importance of knowing more about the institutions 
they administer. ... We will learn more about them because we 
cannot afford the alternative of operating in the dark." 
(Richardson, 1985, p. 8) I second that ern.oticn. 
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